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he  TEMP,  or  Test  and  Evaluation 
Master  Plan,  is  the  framework 
for  an  acquisition  program's 
Test  and  Evaluation  (T&E)  pro¬ 
gram.  The  TEMP  is  a  four-part 
critical  program  document  that 
links  directly  to  the  Acquisition 
Strategy  and  the  System  Perfor¬ 
mance  Specification.  The  TEMP 
shows  how  the  program  will 
verify  and  validate  the  system 
requirements,  whereas  the  Ac¬ 
quisition  Strategy  speaks  to  the 
management  of  the  acquisition 


Conroy  is  a  professor  of  systems  engineering  (test  and  evaluation)  in  the  Capital  and  North¬ 
east  Region  of  the  Defense  Acguisition  University  at  Fort  Belvoir,  Va. 
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of  the  requirements,  and  the  System  Performance  Specifica¬ 
tion  guides  the  development  of  those  requirements.  The  TEMP 
has  a  crucial  role  in  ensuring  that  the  system  meets  the  users' 
requirements  and  capabilities. 

Each  of  the  TEMP's  four  parts  is  integral  to  answering  the 
"why"  questions  surrounding  the  programming  and  planning 
for  the  developmental  test  (DT)  and  operational  test  (OT)  and 
evaluation  methods  and  resources.  If  the  TEMP  is  written  cor¬ 
rectly,  the  order  of  the  four  parts  also  tells  a  story  and  answers 
these  "why"  questions  effectively.  If  these  "why"  questions  are 
used  when  creating  a  TEMP,  it  will  be  a  very  useful  document 
for  managing  the  test  program. 

Parti 

The  TEMP  has  four  main  parts.  Part  I  of  the  TEMP  is  called 
the  Introduction,  but  in  reality  it  is  everything  one  needs  to 
know  about  the  system  being  developed,  tested,  and  evalu¬ 
ated.  The  relevant  question  answered  by  the  information  in 
Part  I  is  "Why  is  this  system  needed?"  One  can  see  that  Part  I 


delve  into  this  part,  let's  talk  a  little  bit  about  the  two  views  of 
testing  in  terms  of  evaluated  performance  and  the  two  views 
of  testing  in  terms  of  evaluation  focus. 

When  testing  the  performance  of  a  system,  the  system  is 
tested  and  evaluated  for  effectiveness  and  suitability.  Effec¬ 
tiveness  is  the  ability  of  the  system  to  meet  its  mission  and 
suitability  is  the  ability  of  the  system  to  be  available  to  meet 
its  mission.  That  is  it  in  a  nutshell.  There  are  more  detailed 
definitions,  but  those  are  the  basics.  An  example  would  be 
that  the  effectiveness  of  a  car  is  that  it  has  the  ability  to  get 
you  to  your  destination  within  your  timeframe,  whereas  the 
suitability  of  a  car  is  that  it  is  reliable  and  ready  to  drive  and 
that  it  can  be  driven.  If  the  car  can  get  you  to  your  destinations 
but  you  need  to  change  the  oil  each  trip,  it  may  be  effective 
but  not  very  suitable. 

Having  said  that,  let's  discuss  the  focus  of  test  and  evalua¬ 
tion.  The  two  main  views  of  evaluation  are  from  the  points  of 
developmental  testing  and  operational  testing.  These  views 


At  the  end  of  the  developmental  test 
program,  we  do  not  want  to  know  that  the  system  works 
well  in  a  lab  or  controlled  environment;  we  want  to  know  what  to 
expect  when  operating  the  system  in  the  real  environment. 


answers  this  question  with  the  background  information  about 
the  system  and  what  capabilities  and  requirements  are  neces¬ 
sary  to  achieve  Its  mission.  Part  I  also  uses  this  information  to 
explain  the  rationale  behind  the  prioritization  of  the  capabilities 
and  requirements  for  the  system  by  explaining  the  nature  of 
the  threat  and  how  the  system  combats  it. 

Part  II 

Part  II  of  the  TEMP  is  known  as  the  Test  Program  Manage¬ 
ment  and  Schedule.  This  section  is  very  straightforward  and  it 
answers  the  primary  "why"  question  of  "Why  does  this  testing 
need  to  be  done  now  and  under  this  budget?"  This  is  important 
because  it  will  constrain  the  amount  of  testing  and  evaluation 
that  can  be  done  on  the  program  to  prove  that  the  system 
is  effective  and  suitable  in  meeting  Its  objectives.  We'll  talk 
more  about  what  it  means  to  be  effective  and  suitable  in  Part 
III.  Part  II  sets  the  boundaries  within  which  the  test  program 
needs  to  be  accomplished  successfully.  This  will  be  significant 
when  trying  to  establish  the  best  tradeoffs  between  how  much 
testing  is  desired  and  how  much  testing  is  needed  to  evaluate 
what  can  be  expected  of  the  system's  true  performance  when 
used  in  the  field. 

Part  III 

The  next  section  is  Part  III,  the  Test  and  Evaluation  Strategy. 
This  section  is  the  heart  and  soul  of  the  TEMP.  But  before  we 


used  to  be  very  diverse,  so  much  so  that  what  is  now  Part 
III  once  was  two  separate  sections,  one  for  developmental 
testing  and  one  for  operational  testing.  Both  views  are  now 
integrated  into  Part  III. 

Developmental  testing  focuses  on  giving  you  what  you  asked 
for.  It  answers  the  question  "Did  I  build  it  right?"  Developmen¬ 
tal  testing  is  more  to  the  point  of  meeting  the  requirement,  or 
what  was  asked  for,  while  trying  to  meet  the  needed  capability. 
However,  if  the  needed  capability  was  not  correctly  translated 
into  a  specified  requirement,  then  what  was  asked  for  may  not 
meet  that  need.  This  second  view,  which  answers  the  ques¬ 
tion  "Did  I  build  the  right  thing?"  is  called  validation  and  is 
the  focus  of  operational  testing.  It  Is  easy  to  see  how  the  two 
can  diverge  if  the  translated  need  is  not  fully  resolved  by  the 
stated  requirements.  One  example  may  be  to  state  the  need 
for  a  200-square-foot  room.  If  this  is  the  only  requirement, 
the  requirement  can  be  met,  or  pass  verification  and  thereby 
developmental  testing,  by  any  combination  of  square  footage 
in  the  room  that  totals  200.  However  a  room  that  is  2  feet  wide 
by  100  feet  long  may  not  suit  your  needs  and  would  not  meet 
validation  or  operational  testing. 

One  can  see  how  important  it  is  that  developmental  testing 
and  operational  testing,  or  verification  and  validation,  are 
given  their  due  in  supporting  each  other  to  gain  the  end  user 
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a  system  that  meets  all  the  requirements  and  capabilities  to 
be  both  effective  and  suitable  in  the  field.  To  this  end,  the  op¬ 
erational  test  community  focuses  heavily  on  integrated  test 
and  evaluation.  Integrated  test  and  evaluation  involve  the  in¬ 
tegration  of  developmental  testing  with  operational  testing. 
This  is  accomplished  in  many  ways,  but  one  of  the  best  ways 
is  to  make  developmental  tests  look  and  feel  like  operational 
tests  as  much  as  possible.  At  the  end  of  the  developmental 
test  program,  we  do  not  want  to  know  that  the  system  works 
well  in  a  lab  or  controlled  environment;  we  want  to  know  what 
to  expect  when  operating  the  system  in  the  real  environment. 
To  do  this,  developmental  testing  environments  need  to  be 
instituted  to  the  greatest  extent  possible  to  simulate  increasing 
levels  of  the  operational  environment,  thereby  decreasing  the 
risk  over  the  test  program  on  the  way  to  a  fielding  decision. 

This  is  ultimately  why  there  is  a  single  combined  developmen¬ 
tal  and  operational  test  focus  in  Part  111  to  reach  both  effective¬ 
ness  and  suitability.  Verification  must  work  with  validation,  and 
effectiveness  must  be  balanced  with  suitability.  Part  ill  brings 
all  these  together  to  explain  the  test  and  evaluation  strategy  as 
a  whole  to  include  how  many  tests  it  will  take,  what  methods 
of  test  and  evaluation  are  necessary  for  each  requirement  and 
capability,  and  how  the  complete  program  balances  to  meet 
the  need.  Ultimately,  Part  111  answers  the  "why"  question  of 
"Why  is  this  combination  of  tests  necessary  to  evaluate  the 
system's  performance?" 

Part  IV 

Part  IV  is  the  final  part  of  the  TEMP  and  it  is  called  the  Re¬ 
source  Summary.  This  is  the  point  everything  else  was  leading 
up  to.  This  is  what  gets  the  plan  done.  Part  IV  is  the  description 
of  the  resources  in  terms  of  funding,  test  sites,  and  test  assets 


that  will  be  needed  to  meet  the  test  and  evaluation  strategy 
described  in  Part  IN. 

Part  IV  is  the  end  of  the  document  but  also  a  beginning  in 
terms  of  evaluating  the  TEMP  to  see  if  it  is  effective  as  a  plan¬ 
ning  tool  for  the  program  once  it  has  been  written.  If  you  ask 
"why"  of  each  part,  you  should  be  able  to  find  the  answer  in 
the  previous  part  and  be  able  to  work  your  way  back  through 
the  TEMP  with  all  your  questions  answered.  If  you  ask  "Why 
are  these  resources  in  Part  IV  needed  to  accomplish  this  test 
program?,"  you  should  be  able  to  find  all  the  answers  in  terms 
of  what  tests  depend  on  those  resources  in  Part  IN.  If  you  ask 
"Why  are  these  tests  constrained  the  way  they  are  in  Part  1 1 1?," 
you  should  be  able  to  find  those  answers  in  Part  11.  And  if  you 
ask  "Why  do  these  tests  in  Part  1 1 1  need  to  be  conducted?,"  you 
should  be  able  to  find  those  answers  in  Part  1.  Finally,  if  you  ask 
"Why  is  the  program  constrained  the  way  it  is  in  Part  11?,"  you 
should  be  able  to  find  those  answers  in  Part  1. 

Summary 

The  four-part  TEMP  is  an  effective  tool  in  planning  the  test 
and  evaluation  program  for  a  system  in  development.  The 
TEMP  has  a  crucial  role  in  ensuring  that  the  system  meets  the 
users'  requirements  and  capabilities  that  are  documented  in 
the  System  Performance  Specification  and  acquired  and  man¬ 
aged  through  the  Acquisition  Strategy.  It  is  a  document  that 
answers  a  number  of  questions  about  the  nature  of  the  test 
and  evaluation  program.  In  answering  those  questions  while 
developing  the  TEMP,  the  TEMP  becomes  more  effective  as  a 
management  and  planning  tool  supporting  the  entire  system 
acquisition  and  management  program.  When  it  comes  to  the 
TEMP,  it  is  OK  to  keep  asking  "why." 

The  author  can  be  contacted  at  Tom.Conroy@dau.mn. 
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